Testing Hypotheses with Induction Trees

Data mining has brought a new philosophy in data analysis that is primarily driven by computational efficiency and predictive performance. In this paper we attempt to show that while this new philosophy opens new horizons, these new techniques may themselves gain much when coupled with traditional statistical reasoning. For example, I recently introduced some sociologists of the University of Geneva to induction trees. These social scientists were impressed by the ease with which such tools allowed them to extract valuable knowledge from their datasets. However, since they were used to fit statistical models like linear or logistic regressions or even multinomial log-linear models, they naturally wanted to know how well the induced trees fit their data. They also wanted to test the significance of specific branch expansions and compare them with alternatives that they found more meaningful. The classification error rates let them unsatisfied. Being primarily interested in how the predictors jointly affect the distribution of the response variable rather than in classification, they expected indeed some divergence Chi-square statistics and inferential tools for comparing alternative structures. Unfortunately they did not found such information in the software outcomes. It is indeed characteristic of data mining and especially of machine learning to focus on the usefulness and predictive performance of the induced rules and to neglect somehow their descriptive content. Thus, the rules are most often used as black boxes. They provide, however, as our sociologists discovered it, also very useful descriptive knowledge about the phenomenon under study. It makes then sense to statistically validate the description provided. Only few attention has been given so far to this aspect. Textbooks, like Han and Kamber (2001) for example, don’t mention it, and, as far as prediction rules are concerned, statistical learning (see Hastie et al., 2001, chap. 7) concentrates on the statistical properties of the classification error rate. This lack of inferential tools for the descriptive content of classification rules motivated this paper. Focusing on induction trees with categorical variables, we propose a simple trick that permits to apply to them the inferential tools used for instance in the statistical log-modeling of multinomial cross tables. The paper is organized as follows. Section 2 discusses the fit issue and introduces the trick that renders induced trees conformable with the requirements of Chi-square statistics. Section 4 is devoted to tree comparison and shows how tests of hypotheses about the tree structure can be carried out with the deviance statistic. Section 5 provides concluding remarks.